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Abstract 

This paper is concerned with constructing an optimal controller in the coherent 
quantum Linear Quadratic Gaussian problem. A coherent quantum controller is 
itself a quantum system and is required to be physically realizable. The use of co- 
herent control avoids the need for classical measurements, which inherently entail 
the loss of quantum information. Physical realizability corresponds to the equiv- 
alence of the controller to an open quantum harmonic oscillator and relates its 
state- space matrices to the Hamiltonian, coupling and scattering operators of the 
oscillator. The Hamiltonian parameterization of the controller is combined with 
Frechet differentiation of the LQG cost with respect to the state-space matrices to 
obtain equations for the optimal controller. A quasi-separation principle for the 
gain matrices of the quantum controller is established, and a Newton-like iterative 
scheme for numerical solution of the equations is outlined. 
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1. Introduction 

Sensitivity to observation is an inherent feature of quantum mechanical sys- 
tems whose state is affected by interaction with a macroscopic measuring device. 
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This motivates the use of coherent quantum controllers to replace the classical 
observation-actuation control loop by a measurement-free feedback, which is or- 
ganized as an interconnection of the quantum plant with another quantum system. 
If such a controller is implemented using quantum-optical components (for ex- 
ample, optical cavities and beam splitters) mediated by light fields [Q], then it is 
dynamically equivalent to an open quantum harmonic oscillator, which consti- 
tutes a building block of quantum systems described by linear quantum stochastic 
differential equations (QSDEs) flS]. 

This leads to the notion of physical realizability which imposes quadratic con- 
straints on the state-space matrices of the controller [0,0, thus complicating the 
solution of quantum control problems which are otherwise reduced to appropri- 
ate unconstrained problems for an equivalent classical system. The links between 
classical control problems and their quantum analogues are known, for example, 
for Linear Quadratic Gaussian (LQG) and Hoo-control. 

The Coherent Quantum LQG (CQLQG) problem seeks a physically realizable 
quantum controller to minimize the average output "energy" of the closed-loop 
system per unit time. This problem has been addressed in [@], where a numeri- 
cal procedure was proposed for finding suboptimal controllers to ensure a given 
upper bound on the LQG cost. Instead, the present paper focuses on necessary 
conditions for optimality and second order conditions for local strict optimality of 
a physically realizable controller and computation of the optimal controller. Both 
approaches make use of the fact that the CQLQG problem is equivalent to a con- 
strained LQG problem for a classical plant, with the LQG cost computed as the 
squared H2-norm of the system in terms of the controllability and observability 
Gramians satisfying algebraic Lyapunov equations. 

We utilize a Hamiltonian parameterization that relates the state- space matrices 
of a physically realizable controller to the free Hamiltonian, coupling and scatter- 
ing operators of an open quantum harmonic oscillator [li]. To obtain equations for 
the optimal quantum controller, we employ an algebraic approach, based on the 
Frechet differentiation of the LQG cost with respect to the state-space matrices 
from I112ll and similar to 111 ill . The resulting equations for the optimal controller 
involve the inverse of special self-adjoint operators on matrices that requires the 
use of vectorization Their spectral properties play an important role in the 
present study. 

Although the optimal CQLQG controller does not inherit the control/filtering 
separation principle of the classical LQG control problem, a partial decoupling of 
equations for the gain matrices still holds. This quasi- separation property leads to 
a Newton-like scheme for numerical computation of the quantum controller that 
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involves the second order Frechet derivative of the LQG cost which is related to 
the perturbation of solutions to algebraic Lyapunov equations. 

The paper is organised as follows. Section[2]specifies the quantum plants being 
considered. Sections [3] and |4] describe physically realizable quantum controllers. 
Section [5] formulates the CQLQG control problem. Sections [6] and [7] introduce 
auxiliary classes of matrices and self-adjoint operators. Section [8] obtains equa- 
tions for the optimal CQLQG controller. Section|9]discusses the quasi-separation 
property. Section [10] establishes a second order condition of optimality. Sec- 
tion \TT\ outlines a Newton-like scheme for computing the optimal controller. Ap- 
pendices provide a subsidiary material on invertibility of the special self-adjoint 
operators, perturbations of inverse Lyapunov operators and Frechet differentiation 
of the LQG cost. 

2. Quantum plant 

We consider a quantum plant with an n-dimensional state vector x t , ap- 
dimensional output y t and inputs w t , rjt of dimensions mi, m 2 . The state and 
the output are governed by the QSDEs: 



Here, A e R nxn , B k e R nxm ", C e R pxn , D e R pxmi are constant matrices, 
and z t is a "signal part" of y t . The state dimension n and the input dimensions 
mi, m 2 are even: n = 2v, = 2/Xfe. The plant state vector x t is formed by self- 
adjoint operators (similar to the position and momentum operators) and, in the 
Heisenberg picture of quantum mechanics, evolves in time t. The entries of the 
mi -dimensional vector w t are self-adjoint quantum Wiener processes [0] whose 
infinitesimal increments compose with each other according to the Ito table 



Here, F is a complex positive semi-definite Hermitian matrix which, on the right- 
hand side of ©, is a shorthand notation for F <8> X, with X the identity operator 
on the underlying boson Fock space and ® the tensor product. We assume that 
vectors are organized as columns unless indicated otherwise, and the transpose 
(•) T acts on vectors and matrices with operator- valued entries as if the latter were 



dx t = Ax t dt + Bidw t + B 2 dri tj 
dy t = z t dt + Ddw t , 
z t = Cxt- 



(1) 
(2) 
(3) 



dwtdwj = Fdt. 



(4) 
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scalars. Also, (-)t := ((-) # ) T denotes the transpose of the entry-wise adjoint 
(•)*. Associated with the Hermitian matrix F from © are real matrices S := 
(F + F)/2 = ReF and T :— (F — F)/i = 2ImF, where (7, Re(-) and Im(-) 
are the entry-wise complex conjugate, real and imaginary parts, and i := a/^T 
is the imaginary unit. The symmetric matrix S contributes to the evolution of 
the covariance matrix of the plant state vector x t , whilst T is antisymmetric and 
affects the cross-commutations between the entries of x t through [dw t , dwj] := 
dw t dwj — (dw t dwJ) T = (F — F T )dt = iTdt. Here, the commutator [a, (3] := 
a/3 — (3a applies entry-wise, and the relation F T = F is ensured by F = F*. In 
what follows, it is assumed that S = I mi , and T is canonical in the sense that 



T:=l 



/a 



1 
-1 



(5) 



where I r is the identity matrix of order r. That is, T is a block diagonal matrix 
with Hi copies of J over the diagonal. By permuting the rows and columns, the 
matrix T from © can be brought to an equivalent canonical form 



T 



'in 



l t'i 







(6) 



where r denotes the (r x r)-matrix of zeros. The canonical antisymmetric matrix 
J of any order satisfies J 2 = —I. Quantum Wiener processes will be assumed to 
have the canonical Ito matrix F — I + i J/2. 



3. Coherent quantum controller 

A measurement-free coherent quantum controller is another quantum sys- 
tem with a n-dimensional state vector £ f with self-adjoint operator- valued entries 
whose interconnection with the plant ©-© is described by QSDEs 

d& = a£ t dt + bidu t + b 2 dy t , (7) 
drjt = ( t dt + duj t , (8) 
Ct = & (9) 

Here, a e W lXn , bi G R nxm2 , b 2 G M" xp , c G R m2Xn , and oo t is a m 2 -dimensional 
vector of self-adjoint quantum Wiener processes which commute with the plant 
noise w t in (OQ) and ©. The combined set of equations CQ)-© and ©-© de- 
scribes the fully quantum closed-loop system in Fig. [U whose output observables 
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Figure 1 : The quantum closed-loop system described by (JTJ — <[3j and ©-Q, where the plant and 
controller noises w and u> are commuting quantum Wiener processes. 



form a po -dimensional process 

Zt = C x t + D Cu (10) 
where Co G M PoXn and D E ]RPoxm 2 are gj ven matrices. The 2n-dimensional 
combined state vector X t := [xj and the output Z t of the closed-loop system 
are governed by the QSDEs 

dX t = AXtdt + BdW t , Z t = CXf (1 1) 

Here, the combined quantum Wiener process W t := [wj ojJ] t has a block di- 
agonal Ito table. The matrices A, B, C of the closed-loop system (fTTI) are given 
by 



(12) 
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where 



b := [6a b 2 ] , B := [B, B 2 ] , C := 



D := 



/ 
D 



(13) 



The dependence of A, B, C on the controller matrices a, b, c is equivalently de- 
scribed by 



T := 



A B 
C 



r + Ti7r 2 , 



7 - 



a b 
c 



(14) 



The affine map j \-t T is completely specified by the plant ®-® through the 
matrices 



A B 
n 
Co 



r i: = 



B 2 

In 

Do 



I n 

COD 



(15) 
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Using the terminology introduced formally in Section [7J the map 7 >->■ r^I^ in 
(PT41) is a grade one linear operator fTi, r 2 |. 



4. Physical realizability 

A controller ©-© is called physically realizable (PR) 0,0] > if its state-space 
matrices satisfy 

aJ + J a T + 6J6 T = 0, b x = J c T J 2 . (16) 

Here, J is a block-diagonal matrix, partitioned in conformance with the matrix b 
from (TOT ) as 



J := D 



Ji 
J 2 



D 



T 



J 2 

£)Ji£) T 



(17) 



and J , Ji, J 2 are fixed real antisymmetric matrices of orders n, mi, m 2 , which 
specify the commutation relations for the controller state variables £ t and the plant 
and controller noises w and u. For convenience, J , Ji, J 2 are assumed to have 
the canonical form © or ©. The relations ( [TBI describe the equivalence of the 
controller to an open quantum harmonic oscillator and the possibility of its quan- 
tum optical implementation [0]. The first of these equations is the condition for 
preservation of the canonical commutation relations for the state variables of the 
quantum harmonic oscillator. The second PR condition, which relates the ma- 
trices b\ and c by a linear bijection, describes the unitary transformation of the 
quantum Wiener process at the input of the quantum harmonic oscillator. The 
first of the PR conditions (fTBI) . which is a linear equation with respect to a, deter- 
mines a as a quadratic function of b up to the subspace of Hamiltonian matrices 
{a G IR nxn : aJ + J « T = 0} = J S„ = §„ Jq, with §„, the subspace of real 
symmetric matrices of order n: 

a= J R + bJb T J /2. (18) 

Hamiltonian matrix particular solution 

Here, R E S n specifies the free Hamiltonian operator t^R£ t /2 of the quantum 
harmonic oscillator [Q], Eqs. (20)-(22) on pp. 8-9]. Since the matrix bJb T is anti- 
symmetric, bJb T Jq is skew-Hamiltonian. Therefore, (fT8~l) describes an orthogonal 
decomposition of the matrix a into projections onto the subspaces of Hamiltonian 
and skew-Hamiltonian matrices in the sense of the Frobenius inner product of 
real matrices (X, Y) := Ti{X T Y), with ||X|| := y/(X,X) the Frobenius norm. 
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From the second PR condition in (fT6l ) and the canonical structure of J and J 2 , it 
follows that the matrix c is related to 6 X by 



I 



(19) 



where, in view of (fT3l ), the matrix I "extracts" bj from b as b\ = 61. In combination 
with the decomposition < fT8l) , this implies that, for a physically realizable quantum 
controller, the matrix 7 in (fT4l) is completely parameterized by the matrices i? and 
6 as 

J n R + bJb T J /2 6] 
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J 2 I T 6 T J 



(20) 



In view of the physical meaning of R, we will refer to (1201) as the Hamiltonian 
parameterization of the coherent quantum controller, with the S n x ]R nx ( m 2+p)- 
valued parameter \R 6] ; see Fig. [2J The PR conditions (fT6l) are invariant un- 




Figure 2: This directed acyclic graph describes the dependence of the LQG cost E of the closed- 
loop system on the matrices R and b. An oriented edge (§H>@ signifies "/3 depends on a". The 
dashed lines encircle the matrix triples 7 and T defined by ( fT4l ). The emergence of R and the 
dependencies indicated by double arrows represent the PR conditions for the quantum controller, 
with a, b, c being otherwise independent. 



der the group of similarity transformations of the controller matrices (a, 6, c) h-» 
(aaa -1 ,ab,ca~ l ), where a is any real symplectic matrix of order n (that is, 
aJ a T = Jn). This corresponds to the canonical state transformation £ t H- cr£ t ; 
see also 11 1 0L Eqs. (12)— (14)]. Any such transformation of a physically realiz- 
able controller leads to its equivalent state- space representation, with the matrix 
R transformed as R a^Ra^ 1 . 
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5. Coherent quantum LQG control problem 

The Coherent Quantum LQG (CQLQG) control problem |@] consists in mini- 
mizing the average output "energy" of the closed-loop system (fTTI) : 



E :-- 



- lim 

-Tt(B t QB) 



E(ZjZ s )ds 
-2(A,H) - 



Tr (CPC 1 



mm . 



(21) 



The minimum is taken over the n-dimensional controllers ©-© which make 
the matrix A in (PT21) Hurwitz and satisfy the PR conditions < fT6l) . Here, EX : = 
Tr(pX) denotes the quantum expectation over the underlying density operator p, 
and P := lim^ +00 ReE(X t X^) is the steady-state covariance matrix of the state 
vector of the closed-loop system. Also, we use the shorthand notation 



H := QP, 

with P and Q satisfying the algebraic Lyapunov equations 



AP + PA T + BB V = 0, A L Q + QA + C l C = 0, 



T/ 



7T/ 



(22) 



(23) 



so that these matrices are the controllability and observability Gramians of the 
state-space realization triple (A, B, C). The spectrum of the diagonalizable matrix 
H in (1221) is formed by the squared Hankel singular values of the system, and 
we will refer to H as the Hankelian. The fact that E coincides with the squared 
H2-norm of a classical strictly proper linear time invariant system enables the 
CQLQG problem (l2~TT) to be recast as a constrained LQG control problem for an 
equivalent classical plant 
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(24) 



driven by a (m 1 + m 2 ) -dimensional standard Wiener process, with the controller 
being noiseless. We will employ the smooth dependence of the cost E on the 
matrices R and b which govern the Hamiltonian parameterization (1201) of a phys- 
ically realizable stabilizing controller. The conditions of optimality, obtained in 
Section [H utilize the Frechet differentiation of the LQG cost with respect to the 
state- space realization matrices lll2n assembled into matrices with a specific spar- 
sity pattern and an auxiliary class of self-adjoint operators introduced in Sections[6] 
and H 
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6. The r sparsity structure 



The subsequent considerations involve Frechet differentiation with respect to 
state-space realization matrices assembled into matrices of the 'T-shaped" spar- 
sity structure (PT4l) . We denote by 



r,m,p •- 



(f a 

T 



: if G R rx >6 



r G K p> 



(25) 



the Hilbert space of real (r + p) x (r + m) -matrices whose bottom-right block 
of size (p x m) is zero. The space T r ^ p , which is a subspace of ]R( r +p) x ( r + m ) j 
inherits the Frobenius inner product of matrices. Let Ti. r ^ p denote the orthogonal 
projection onto r rjm p whose action on a (r + p) x (r + m) -matrix consists in 
padding its bottom-right (p x m)-block ip with zeros: 



n 



r,m,p 



( 




a 


)- 




a 




r 






r 






(26) 



The subscripts in r rjmjP and n r m p will often be omitted for brevity. The Frechet 



derivative d x f of a smooth function T 3 



if a 

T 



x h. /(X) e 



be- 



longs to the same Hilbert space (T23T) and inherits the sparsity structure: dxf 

Orf 



7. Special self-adjoint operators 

For the purposes of Section[H we associate a linear operator [a, /3| : W xq — >■ 
]R SX ' with a pair of matrices a e R sxp and /3 G M 9X ', by 

!«,/?! (X) :=aXj?. (27) 

The map h-> from the direct product of the matrix spaces to the 

space of linear operators on matrices is bilinear. If s — p and t = q, then the 
spectrum of the operator fla, /3| on M px<? consists of the pairwise products Xjfik 
of the eigenvalues Ai, . . . , A p and /ii, . . . , p, q of the matrices a and (3, so that their 
spectral radii are related by 

r([ a ,/3j) = r(a)r(/3). (28) 
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Furthermore, for any positive integer r and matrices a\, . . . , a r G ~R sxp and 0i, . . . , (3 r G 
R. qxt , we define a linear operator 

r 

[oti,/3i | ... | Or^rl := (29) 

fe=l 

where the matrix pairs are separated by "|"s. Of importance will be self-adjoint 
linear operators on the Hilbert space W xq of the form d29l where ai, . . . , a r G 
M pxp and (3i,. . . ,(3 r G M 9X9 are such that for any k = 1, . . . , r, the matrices 
«fc and are either both symmetric or both antisymmetric. Such an operator 
(|29l will be referred to as a self-adjoint operator of grade r. The self-adjointness 
is understood in the sense of the Frobenius inner product on M. pxq and follows 
from the property that, in each of the cases (a T ,(3 T ) = (±q, ±/3), the adjoint 
fa, (3^ = fa T , /3 T | coincides with fa, In these cases, as for any self-adjoint 
operator, the eigenvalues of flee, /3| are all real. 

Lemma 1. If a G M. pxp and (3 G M. qxq are both antisymmetric, then the spectrum 
of fla, /3| is symmetric about the origin. If a and (3 are both symmetric and positive 
(semi-) definite, then fa, /3| is positive (semi-) definite, respectively. 

Proof. If a and (3 are both antisymmetric, then their eigenvalues Ai, . . . , X p and 
Hi, . . . , fx q are all pure imaginary and symmetric about the origin yfl. Hence, the 
eigenvalues XjHk of fa, /3| also form a set which is symmetric about the origin. 
By a similar reasoning, if a and (3 are real positive (semi-) definite symmetric 
matrices, then their eigenvalues are all real and (nonnegative) positive, and hence, 
so are the eigenvalues of fa, /3| which implies its positive (semi-) definiteness. 
Alternatively, the second assertion of the lemma also follows from the relation 
fa, /3| = f\fa, v^fl 2 which holds for any positive semi-definite symmetric ma- 
trices a G W xp and (3 G R qxq , so that (X,aX/3) = \\^X^\\ 2 ^ for any 
X G W xq . □ 

Whilst the operator (ITTT) with nonsingular a and (3 is straightforwardly invert- 
ible: fa,P\~ l = fa~\ /3~% the inverse of M := fa u fii | ... | a r ,f3 r l from 
dH for r > 1 (except for the case £\ k faj, /3J = fl^. aj, J2 k Pk\, which re- 
duces to a grade one operator, or special Lyapunov operators fa, J| + fl, aj with 
a = a T which are treated by diagonalizing the matrix a), can only be computed 
using the vectorization of matrices [5] as AA^iY) = vec _1 (S _1 vec(y)), pro- 
vided that the matrix H := J2k=i flu ® a k i s nonsingular. Here, vec : M. pxq — > W q 
is a linear bijection which maps a matrix X to the vector obtained by writing 
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the columns X 9l , . . . , X, q of the matrix one underneath the other. Invertibility 
conditions for grade two operators is discussed in | Appendix A 



8. Equations for the optimal controller 



Necessary conditions for optimality in the class of 
n-dimensional physically realizable stabilizing controllers are obtained by equat- 
ing the Frechet derivatives of the LQG cost E with respect to R and b to zero. In 
view of Fig. [21 the chain rule allows the differentiation to be carried out in three 
steps. First, the matrices A, B, C of the closed-loop system are considered to be 
independent variables. Below is an adaptation of 11121 . Lemma 7 of Appendix B] 
whose proof is given to make the exposition self-contained. 

Lemma 2. Suppose the matrix A in 4721) is Hurwitz. Then the Frechet derivative 
of the LQG cost E from rf2~7T) with respect to the matrix Y from (QU) is 



d T E 



H QB 
CP 



(30) 



Here, H is the Hankelian defined by ([22]) in terms of the Gramians P, Q from 

Proof. As discussed in Section [6l the Frechet derivative dpE inherits the block 
structure of the matrix T: 



d r E 



d A E d B E 
d c E 



(31) 



We will now compute the blocks of this matrix. To calculate d A E, let B and C 
be fixed. Then the first variation of E with respect to A is SE = (C T C, SP) = 
-(A T Q+QA,5P) = -(Q,A5P+(5P)A T ) = (Q, (5A)P+P5A T ) = 2(H,5A), 
which implies that 

d A E = IE. (32) 

To compute 8qE, suppose A and C are fixed. Then the observability Gramian Q, 
which is a function of A and C, is also constant, and the first variation of E with 
respect to B is 5E = (Q,5(BB T )) = (Q, (5B)B T + B5B T ) = 2(QB,6B), and 
hence, 

d B E = 2QB. (33) 

The derivative dc E is calculated by a similar reasoning. Assuming A and B (and 
so also the controllability Gramian P) to be fixed, the first variation of E with 



11 



respect to C is 5E = (P,5(C T C)) = (P, (5C) T C + C T 5C) = 2(CP,8C), which 
implies that 

d c E = 2CP. (34) 

Now, substitution of (I32l)-(l3"4l) into (ED yields (l3~0l> . □ 

We will now take into account the dependence of the closed-loop system ma- 
trices A, B, C in (fT2l) on the controller matrices a, 6, c, with the latter still con- 
sidered to be independent variables. In what follows, the Gramians P and Q in 
(1231 and the Hankelian if, defined by (1221) . inherit the four-block structure of the 
matrix A from (fT2l) . Their blocks have size (n x n) and are numbered as follows: 



H :-- 



H21 H22 



■J- n 



<— 2n— > 

H 2 . 



(35) 



The block (-)n is related to the state variables of the plant, while (-) 22 pertains to 
those of the controller. The blocks of the matrix H in (1351) are expressed in terms 
of the block rows of Q and block columns of P as Hjk = Qj.P,k- 

Lemma 3. Suppose the matrix A in 4721) is Hurwitz- Then the Frechet derivative 
^d a E d b E~\ 



d^E 



d r E 



d^E 



of E from rt271) with respect to the matrix^ from ([74]) is 

Q 2 .i3D r 



H 22 H21 
BjH 12 + DJCP.2 







(36) 



where the matrices r\, T 2 are defined by ( 1771 ) ; if, P, Q are given Z?y d22l)-(|23|), 
and ?/ze notation (TJJJ) is nsed. 



Proof Since P is a composite function of a, b, c which enter (1211) through the 
closed-loop system matrices A, B, C, the chain rule gives 



d y E = {d^T)\drE) = U(T^drET^ 



(37) 



Here, (-)t is the adjoint in the sense of the Frobenius inner product of matri- 
ces, and II is the orthogonal projection onto the subspace T defined by (1251)- 
(1261) . Indeed, the first variation of the affine map 7 4 T, defined by (fT4l)- 
(p3>. is given by 5r = ^((fy)^, which implies that d^T = |Ti,r 2 l. Hence, 
5E = (d r E,5T) = (drE^rz) = (TjdrETj^^) = {n(TjdrET^),5^, 
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which establishes (|37l) . Substitution of the matrices I\ and T 2 from (fT5l) and <9r-E 
from ( 1301 ) into the right-hand side of (1371) yields 



<9 7 £ = 2n 











5 2 T Dq 



H QB 
CP 



C T 

In 

D T 

H 22 H 21 C T + Q 2 .£D T 

BjH 12 + L£CP. 2 

where Lemma[2]and the notation (|35l) are also used, which proves ( |36l ). 



□ 



Finally, we will utilize the Hamiltonian parameterization (1201) . which makes 
E a function of the matrices R and b; see Fig. [2] 

Theorem 1. A physically realizable stabilizing controller, with Hamiltonian pa- 
rameterization rf20l) , 15 a critical point of the LQG cost E from ([27]) if and only if 
there exists a real antisymmetric matrix $ of order n such that 



H 22 = -$J , 

m(b) + H 21 C T + Q 21 BB T 

+J (Hj 2 B 2 + P 21 C^D )J 2 I T = 0. 



Here, 



J | Q 22 , DD T | J P 22 J Q ,lJ 2 D^D J 2 i ] 



(38) 
(39) 

(40) 



is a self-adjoint operator of grade three in the sense of<[29\). 



Proof. In view of ( |20l) . the symmetric matrix R enters the controller only through 
a. Hence, 



d R E = (-J d a E + (- J d a E) T )/2 = Hgj - J H 22 , 



(41) 



where the relation d a E = 2H 22 from Lemma [3] is used. Unlike R, the matrix b 
both enters a and completely parameterizes c, and hence, 

dE/db =((d a E)J + J (d a E) T )bJ/2 + d b E 
+ J (d c E) T J 2 I T 
=(H 22 J + J Hj 2 )bJ + 2(H 21 C T + Q 2 .BB T ) 

+ 2J (BjH 12 + D^CP. 2 ) T J 2 1 T , (42) 
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where ( 1361 ) of Lemma[3]is used again. By introducing a real antisymmetric matrix 

$ := (H 22 J + JoHj 2 )/2, (43) 

and recalling (fl2)). (TT3T) and ([35]), it follows from (02]) that 

(d£?/d6)/2 =$6J + # 2 iC T + Q 2 iPD T + Q 22 6DD T 
+ J Q (H? 2 B 2 + P 2 iC T P ) J 2 I T 
+ J P 22 J MJ 2 P T A) J 2 I T 
=H 21 C T + Q 21 BB T 

+ J (H? 2 B 2 + P 21 C T P> ) J 2 I T + mt(b), 

where ( TT9b and (l40l) are also used. Therefore, dP/d& = is equivalent to ( l39l) . 
The definition (|43~T) . which is considered as an equation with respect to P 22 , de- 
termines uniquely the skew-Hamiltonian part — $ J of P 22 , so that H 22 can be 
represented as 

P 22 = J , (44) 

where 

* := (JoHg - H 22 J )/2 (45) 
is a real symmetric matrix of order n. Direct comparison of (1431) with (I4TI) yields 

<9 fl P = -2J ^J . (46) 

Hence, <9#P = holds if and only if ^ = 0, in which case, (|44|) takes the form 
of (1381) . Therefore, the property that the controller is a critical point of E (that is, 
d R E = and dE/db = 0) is indeed equivalent to the fulfillment of ([38]) and d3~9l) 
for a real antisymmetric matrix $ of order n. □ 

For a given matrix b in the Hamiltonian parameterization (l20l) of the controller, 
(1451) defines a map R(6) 3 R H> * e S n on the set 

R(6) := {P G §„ : 4 is Hurwitz}. (47) 

In view of (l46l) . the Frechet derivative of this map with respect to P is expressed 
in terms of the second order Frechet derivative of the LQG cost of the closed-loop 
system as 

d R * = ~lJ ,J ]dt l R E 1 (48) 

where we have also used the property that [ J , Jo] is involutory since f J , Jo]] 2 = 
[Jo , JqI = I-/, -1} is the identity operator. 
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9. A Quasi-separation principle 

The operator Wl, which is defined by (140b and acts on the controller gain matrix 
b from (fT3l) . can be partitioned as 

£DT(6) = [Wli(h) D)l 2 (b 2 )] (49) 

into two operators acting separately on the submatrices b\ and b 2 . Here, 

S»li : = !*, ^2 | Q 22 , / I Jo, ^A^o J 2 1, (50) 

0« 2 :=[$,DJiD T |Q 2 2,DI> T l (51) 

are self-adjoint operators of grades three and two. This allows the equation ( 139b 
for dE /db = to be split into 

ORifo) + Q 2 iP 2 + J (^ 2 B 2 + P 21 C T J D ) J 2 = 0, (52) 
Tl 2 (b 2 ) + H 21 C T + Q^B^ = 0, (53) 

which are equivalent to dE/dbi = and dE/db 2 = 0. Note that (|52b corresponds 
to the equation for the state-feedback matrix 

c = -(D^D )~ 1 (BjQ 1 + DfCo) (54) 

of the standard LQG controller for the subsidiary classical plant (124b . while (l53b 
corresponds to the equation for the Kalman filter observation gain matrix of the 
controller 

b 2 = (P!C T + B l D r )(DD T )- 1 . (55) 

Here, it is assumed that the matrix D is of full column rank, and D is of full row 
rank. The matrices c and b 2 from (|54b and (155b determine the dynamics matrix of 
the standard LQG controller as a := A — b 2 C + B 2 c and are expressed in terms of 
the stabilizing solutions Qi, Pi of the independent control and filtering algebraic 
Riccati equations (AREs): 

A T Qi + QiA + C T C 

= (QxB 2 + C^D )(D^D )- 1 (Q 1 B 2 + C T P ) T , 
APi + P X A T + P x pT 

= (AC 7 + B l D T ){DD T y\P 1 C r + P!P T ) T . 
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The fact, that (|52l) and (1531) are independent linear equations with respect to b± 
and b 2 , as well as the original partition (|49l ), can be interpreted as an analogue of 
the classical LQG control/filtering separation principle for the CQLQG problem. 
In turn, each of the operators VJlk from (l50l) and (15TI) can be split into the sum of 
self-adjoint operators 9Jt£ and Q7t£ of grades one and less one: 




By applying LemmaCD it follows that the spectrum of 9Jt£, is symmetric about the 
origin, while Wl^ y 0. Moreover, if Q22 >- 0, or P 22 y and D in (TTOl) is of 
full column rank, then Wlf y 0. Indeed, the fulfillment of at least one of these 
conditions implies positive definiteness of at least one of the positive semi-definite 
operators on the right-hand side of the representation 

Tl+ = [Q22, 1} + P0P22JJ, ^DoJJl (58) 

which follows from J and J 2 being antisymmetric matrices. Similarly, the con- 
ditions that Q 22 y and D is of full row rank ensure that yjl 2 y 0. In particular, 
by adapting [|12l Lemma 5 of Section VIII], it follows that if, in addition to the 
rank conditions on D and D, the controller state-space realization is minimal, 
then Q 22 y and P 22 y and hence, Wl^ >~ and ^2 >~ °- Therefore, in the 
cases discussed above, the invertibility of the operators QJti and Wl 2 in (l56l)-(l57l) 
can only be destroyed by the presence of the indefinite operators and 9Jt 2 if 
the matrix $ is large enough compared to Q 22 . This can be formulated in terms of 
the matrix 

A := (59) 
whose spectrum is pure imaginary and symmetric about zero. 

Lemma 4. Suppose the matrix D in (0) is of full row rank and Q 22 y 0. Also, 
suppose the spectral radius of the matrix A from rt59l) satisfies r(A) < 1. Then 
the operators OJli and dJt 2 in rt50l) and rf571) are positive definite. 

Proof Since [J iWo, </ 2 £>o D o M > 0, and IQ22, >- (in view of the as- 
sumption Q 22 y 0), then (1551) and d5HT) imply that 

^ym^ + lQ^,!} y (1 -r(A))lQ 22) /]. (60) 
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Here, we use the relation r(lQ 22 , /| -1 9DI?) = r(A)r( J 2 ) = r(A) which follows 
from ( |28l ) and the property that the eigenvalues of the canonical antisymmetric 
matrix J 2 are ±i. Therefore, if r(A) < 1, then (l60l) implies that Wli y 0. By 
a similar reasoning, under the additional assumption that D is of full row rank 
(that is, DD T y 0), it follows from $57$ and ([59]) that M 2 ^ (1 - r(A))9J£ >- 0. 
Indeed, r((9H 2 f )^ 1 ^) = r(A)r(DJ 1 D T (DD T )- 1 ) ^ r(A) since -I ^ U x I 
and the Hermitian matrix (DD rT )^ 1 ^ 2 D(iJi)D T (DD T )~ 1 ^ 2 has all its spectrum in 
[-1, 1], so that r{DJ 1 D' T {DD' v )- 1 ) < 1. □ 

Assuming invertibility of the operators 97ti and 37t 2 (for example, the fulfill- 
ment of conditions of Lemma|4]that ensure a stronger property - positive definite- 
ness of these operators), the equations (|52l) and (|53l) can be written more explicitly 
for b\ and 6 2 : 

h = -m^ 1 {Q 2l B 2 + J (f^^2 + P2iC^D ) J 2 ), (61) 
6 2 = -m 2 1 (H 21 C T + gaifii.D 7 ). (62) 

These two equations are, in principle, amenable to further reduction (to be dis- 
cussed elsewhere) and will be utilized as assignment operators in the iterative 
procedure of Section[TT]for finding the optimal controller. 



10. Second order condition for optimality 



A second order necessary condition for optimality of the controller with re- 
spect to the matrix R of the Hamiltonian parameterization (l20l) is the positive 
semi-definiteness d 2 R E y- of the appropriate second Frechet derivative of the 
LQG cost (EU). Moreover, the positive definiteness d\E y is sufficient for the 
local strict optimality. To compute the self-adjoint operator d\E, which acts on 
the subspace §„ of real symmetric matrices of order n, we define a linear operator 
J : § n — > R 2nx2n as an appropriate restriction of the grade one linear operator 
relating A with R: 



J 



n 

Jo 



[On Ir 



(63) 



Its adjoint is J ] = -«S|[0„ J ] 



0„ 



— > §> n the symmetrizer defined by (|B.2b 



since J 1S antisymmetric, with S 
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Lemma 5. Suppose the matrix A in ([72]) is Hurwitz- Then the second Frechet 
derivative of E from rt2"71) with respect to the matrix Rfrom ( T2771 ) is 



d\E = AJ\QC A SV + VC A tSQ)J. 



(64) 



Here, Ca and S are the inverse Lyapunov operator and symmetrizer from ( U?.i|) , 
( lfi.21 ), and Q := and V := |J, P\ are grade one self-adjoint operators 

(see Section^ of the left and right multiplication by the observability and con- 
trollability Gramians Q and P of the closed-loop system from rf23l) . 

Proof. The matrix R only enters the cost E through the matrix A of the closed- 
loop system, and A depends affinely on R, with 8rA = J the constant oper- 
ator from (|63T> . Hence, ([64]) follows from d\E = J ] d\EJ and Lemma [1] of 
Appendix C[ □ 



From ([641) . it follows that the "matrix" representation of the self-adjoint oper- 
ator d R E on the space S n is described by 

vech(<9|£(M)) = 4T T (fi + fi T )Tvech(M), 

where vech(M) denotes the half-vectorization of a matrix M G S n , that is, the 
column-wise vectorization of its triangular part below (and including) the main 
diagonal. Here, the square matrix 



ft := ~{hn ®Q){hn®A + A® J 2 n)" 1 S(P ® I 2n ) 

2nx2n , with E corresponding to 



of order An 2 represents the operator QC^SV on 
the symmetrizer S : R 2nx2n § 2n . Also, 



T := 



( 


'On 














A 


) 



A 



is a (4n 2 x n(n + l)/2)-matrix which represents the operator J , defined by ([63]) , 
with A G R n2xn ( n + 1 )/ 2 the "duplication" matrix [SQj]] which expresses the full 
vectorization of a matrix M G §„ in terms of its half-vectorization by vec(M) = 
Avech(M). 



11. A Newton-like scheme 

The equations ([6Tb— (f62b can be combined with iterations for solving the equa- 
tion ^ = for the matrix ^ from ([45] ), which is equivalent to the stationarity of 



18 



the LQG cost E with respect to the matrix R of the Hamiltonian parameterization. 
The latter part of the scheme, aimed at finding a root R E R(6) of the equation 
^ = from the set (l47l) . can be organized in the form of Newton-Raphson itera- 
tions 

R^R- (dn^y 1 ^) — R - {d 2 R E)-\d R E). (65) 

Here, the symmetric matrices d R E and \1> are related by (|46l , and, in view of ( |48l) , 
the inverse of the operator d R ^> is given by 

{d B *)- 1 = -2{& R E)- x \J< h J Q \ i (66) 

where we have again used the involutional property of the operator fj , </ol> an d 
the second order Frechet derivative d R E is provided by Lemma |5] If the local 
strict optimality condition d R E y is satisfied, this ensures well-posedness of 
the inverse in (l66l) . Thus the equations (l6"TI)-(l62"l). considered as assignment op- 
erators for b]_ and b 2 , and (1631 ) for R, constitute a Newton-like iterative scheme 
for numerical computation of the state- space realization matrices of the optimal 
CQLQG controller. These three assignment operators are alternated with updating 
the Gramians of the closed-loop system via the appropriate Lyapunov equations 
in (|23T) . The order of this alternation will influence the overall convergence rate of 
the scheme and is an important computational issue to be explored. Another issue 
to be taken into account is that the asymptotic stability of the closed-loop sys- 
tem matrix A can be violated by the update of the matrices b\, b 2 , R after which 
the next iteration becomes impossible. Therefore, being a local optimization algo- 
rithm, the proposed scheme requires a "stability recovery" block. A salient feature 
of such an algorithm (which is currently under development) is that it involves the 
inversion of special self-adjoint operators on matrices which, in general, can only 
be carried out via the vectorization of matrices mentioned in Sections 171 and [101 

12. Conclusion 

We have obtained equations for the optimal controller in the Coherent Quan- 
tum LQG problem by direct Frechet differentiation of the LQG cost with respect to 
the pair of matrices which govern the Hamiltonian parameterization of physically 
realizable quantum controllers. We have investigated spectral properties of spe- 
cial self-adjoint operators whose inverse plays an important role in the equations 
and can only be carried out by using matrix vectorization. We have established a 
partial decoupling of these equations with respect to the gain matrices of the op- 
timal controller, which can be interpreted as a quantum analogue of the standard 
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LQG control/filtering separation principle. Using this quasi-separation property, 
we have outlined a Newton-like iterative scheme for numerical computation of the 
quantum controller. The scheme involves a yet-to-be-explored freedom of choos- 
ing the order in which to perform iterations with respect to the Hamiltonian and 
gain matrices of the controller to optimize the convergence rate. The existence 
and uniqueness of solutions to the equations for the state- space realization matri- 
ces of the optimal CQLQG controller also remains an open problem and so does 
their further reducibility. This circle of questions is a subject of ongoing research 
and will be tackled in subsequent publications. 
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Appendix A. Invertibility of grade two operators 

Lemma 6. Let r = 2 in ft29\) , and let both matrices ol\ and 0i be nonsingular. Then 
the operator M. := (3\ | a 2 , fi 2 \ is invertible if and only if the eigenvalues 
Ai, . . . , X p of ol{ x ol 2 and the eigenvalues fi±, . . . , p q of f3 2 f3i 1 satisfy 

Xjfik^-l for all j = l,...,p, k = 1, ...,<?. (A.l) 

Proof. Ifr = 2, the operator (1291 can be represented as M. := |ai,/3i | a^Aj = 
MiM 2 , where Mi := [ai,0il andA^ 2 := | a^a^^i 1 }. The operator 
A4i is invertible in view of the nonsingularity of the matrices a± and 0i, with 
Mi 1 = fa] -1 , /Sf 1 !- Hence, the invertibility of M is equivalent to that of M 2 - 
In turn, the operator Ai 2 is invertible if and only if its spectrum {1 + Xj/^k '■ 1 ^ 
j ^ p, 1 ^ k ^ q] does not contain 0, which is equivalent to (IA. II) . □ 

By Lemma [6[ the nonsingularity of the matrix Y^k=i &k ® a k °f order pq 
reduces to a joint property of individual spectra of two matrices of orders p and q. 
This reduction does not hold for r > 2. 
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Appendix B. Perturbation of inverse Lyapunov operators 



We associate an inverse Lyapunov operator Ca with a Hurwitz matrix A £ 
l nx ™, so that Ca maps a matrix M £ IR nxn to the unique solution N of the 
algebraic Lyapunov equation AN + NA T + M = 0: 



£a(M) :-- 



e At Me ATt dt. 



(B.l) 



Its adjoint is C A = £^ T - Since Ca commutes with the transpose, that is, Ca^M 1 
(Ca(M)) t , then it also commutes with a symmetrizer S defined by 



S{M) := (M + M T )/2. 



(B.2) 



The operator S : W ixn — > E> n is the orthogonal projection onto the subspace of 
real symmetric matrices of order n. 

Lemma 7. The Frechet derivatives of the controllability and observability Grami- 
ans P and Q of an asymptotically stable system (A, B, C) with respect to the 
A B 

are expressed in terms of< \B.l\) and < \B.2\) as 



matrix T :- 



C 



d T P = 2C A Sl[l 0] 
d r Q = 2C A TSl[Q C T ] 



P 
B T 



(B.3) 
(B.4) 



Proof. The Frechet differentiability of P and Q is ensured by the assumption that 
A is Hurwitz. The first variation of the algebraic Lyapunov equation AP + PA T + 
BB T = yields 



= (5A)P + A5P + (5P)A T + P5A T + (5B)B T + B5B T 

P 

B T 



ASP + (5P)A T + 2S^ [6 A SB] ^ T J 



This is an algebraic Lyapunov equation with respect to 5P with the same matrix 
A, which proves (IB .3 b in view of the identity [A £>] = [/ 0] T. The relation 
(IB .41 ) is obtained by a similar reasoning from the first variation of the Lyapunov 
equation for the observability Gramian Q, or by using the duality between P and 
Q. □ 
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Appendix C. Second Frechet derivative of the LQG cost 
Lemma 8. The second Frechet derivative of the squared ^-norm E 
of an asymptotically stable system with respect to the matrix V : = 
computed as 



\U,B,C)\\l 



A B 
C 



is 



d 2 v E =4[ 



+ 41 
+ 2fl 



[P B]]C A tS{[Q C t ] 
[I 0]]C A SI[I 0], 



Q 
/ 




/ 




/ 



I 



p 

B T 

P 
/ 



(C.l) 



Here, La and S are the inverse Lyapunov operator and symmetrizer from rt.fi.il) , 
( \B.2\) , and P, Q are the controllability and observability Gramians of the system. 

Proof. Lemma |2] implies that the first variation of the Frechet derivative drE is 
computed as 



SdrE/2 =5 



QP QB 
CP 



5Q[P B] 



5P[I 0] 



Q5B 
(5C)P 



Hence, (IC.ll ) is obtained by using the Frechet derivatives of the Gramians from 
Lemma |7] of | Appendix B and the identity 





(5C)P 



Q5B 




Q 


5T 





+ 





5T 


P 







/ 


/ 




/ 


/ 



□ 



Lemma 9. The second Frechet derivative of the squared H2-norm E := \\(A, J3,C)||| 
of an asymptotically stable system with respect to A is 



d\E 



ATI, 



K:= QC A ST + T£ A tSQ. 



(C2) 



Here, Q := fQ, /J and V := |J, P] are grade one self-adjoint operators (see 
Section [7|) of the left and right multiplication by the observability and controlla- 
bility Gramians of the system. 
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Proof. In view of Lemma [7J the first variation of d A E = 2QP with respect to A 
is 

6d A E = 2(Q5P + (5Q)P) 

= A(QC A S((5A)P) + C A tS(Q{SA))P) 

which establishes (IC.2I ). Alternatively, (IC.2I ) can be obtained from (IC.ll ) of Lemma[U 

□ 

Note that at least some eigenvalues of the self-adjoint operator 1Z in (IC.2I) are 
positive, since 72. (A) = — QP is the negative of the Hankelian, and (A, TZ(A)) = 
-(A,QP) = \\(A,B,C)\\ 2 2 /2>0. 
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